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ABSTRACT 


Decision  aids,  even  well  designed  ones,  have  demonstrated  a  wide 
range  of  utility  and  effectiveness  when  employed  in  the  operational 
environment.  The  development  and  employment  of  the  PROPHET  HF  (1  - 
50  MHz.)  propagation  assessment  system  has  been  observed  for  over 
a  decade.  Recent  studies  indicate  that  some  of  the  early 
assumptions  on  how  the  module  was  to  be  employed  were  invalid. 
Although  it  uniquely  fills  a  real  need  and  its  models  and  products 
have  been  extensively  tested  in  operational  environments  by  its 
users,  which  include  most  military  and  civilian  branches  of  the  US 
Department  of  Defense,  PROPHET  faces  an  uncertain  future  as  a 
stand-alone  system.  ^Jts  main  strength  is  to  provide  mission 
oriented  and  scenario  specific  products  to  perform  HF  signal 
coverage,  signal  vulnerability  and  radio  circuit  connectivity  in 
near  real  time.  While  technically  sound  in  meeting  these 
objectives,  PROPHET'S  effectiveness  is  limited  by  variations  in 
user  training,  experience,  motivation  and  the  lack  of  operationally 
convenient  access.  These  problems  appear  to  apply  in  some  degree 
to  the  employment  of  most  decision  aids.  Based  on  the  PROPHET 
experience,  this  paper  will  describe  the  fallacies  that  are  common 
in  decision  aid  development  and  employment,  how  they  affect  the  DA 
effectiveness,  a  recommendation  for  the  next  generation  propagation 
assessment  decision  aid  and  results  from  an  initial  demonstration 
prototyping  effort  to  check  out  some  of  the  new  concepts. 

PROPHET:  PAST.  PRESENT  AND  FUTURE 


PROPHET  is  the  Navy's  standard  tactical  HF  prediction  decision  aid. 
It  is  designed  to  provide  real-time  information  about  HF  signals 
warfare  and  the  connectivity  of  radio  communication  circuits.  Since 
its  inception,  PROPHET'S  development  has  emphasized  operational 
deployment.  Its  output  products  have  been  designed,  to  a  large 
extent,  at  the  direct  request  of  its  operational  users. 


PROPHET  is  a  self-contained,  software  system  which  was  conceived 
in  1974  and  first  deployed  in  1976.  It  has  grown  from  an  initial 
5  output  products  to  its  present  25  product  capabilities,  each  of  0I1 
which  has  several  sub-options.  The  most  recent  release,  Version 
3.2,  has  been  provided  to  several  hundred  users.  It  uses  simple 
emulations  of  complex  propagation  phenomenon  to  achieve  the 1 
portability  and  speed  of  personal  computers.  PROPHET  contains  en¬ 
large  amounts  of  empirical  and  intuitive  knowledge  and  user 
experience,  lending  it  the  attributes  of  an  expert  system. 
Throughout  its  development  ar.d  employment,  the  models  in  PROPHET  ,n/ 
have  undergone  extensive  verification  against  "ground  truth"  g£ 
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oblique-sounder  data.  Its  developers  have  maintained  a  continuous 
dialogue  with  the  user  community  to  ensure  continuing  operational 
credibility.  This  has  resulted  in  a  broad  spectrum  of 
applications  and  users. 

Today  PROPHET'S  future  as  a  stand-alone  capability  is  in  doubt. 
In  spite  of  it's  widespread  use  and  usefulness,  PROPHET  has  not 
been  institutionalized  or  achieved  the  programmatic  acceptance 
required  to  sustain,  enhance,  and  promulgate  it.  Further 
development  and  promulgation  remain  an  Ad  Hoc  process.  This 
situation  prompted  PROPHET'S  developers  at  NOSC  to  review  the 
fundamental  philosophy  of  developing  and  implementing  decision  aids 
for  the  military  community.  This  in-house  review  indicated  errors 
were  made  in  the  assumptions  on  how  the  decision  aid  would  be 
normally  employed  and  identified  ways  to  correct  the  problems. 
The  remainder  of  this  paper  summarizes  these  findings,  proposes 
recommendations  for  the  next  DA's,  and  describes  the  results  from 
an  initial  prototyping  effort  to  check  out  some  of  the  new 
concepts. 

MAJOR  FALLACIES  IN  DEVELOPING  TACTICAL  DECISION  AIDS 

Decision  aids  (DA) ,  as  a  generic  set  of  software,  typically  suffer 
from  at  least  two  fallacies  in  their  concept  and  development.  The 
first  fallacy  is  that  when  the  user  interface  is  designed,  the 
developer  assumes  the  end  user  will  have  the  background  to  ask  the 
right  questions  in  the  right  order.  For  example,  when  each 
PROPHET  module  was  conceived,  it  was  assumed  the  user  would  have 
the  training  or  experience  in  HF  signals  warfare  or  frequency 
planning  to  know  the  logical  inquiry  sequence  to  successfully  solve 
the  problem.  Review  of  PROPHET  users  and  the  platforms  where  it  has 
been  used  indicates  that  was  not  a  valid  assumption.  In  fact,  with 
respect  to  HF  operations  in  general,  it  is  more  valid  to  assume  the 
contrary.  In  numerous  operational  tests,  it  has  been  observed  that 
PROPHET  users  span  a  broad  spectrum  in  motivation  and  HF 
experience.  The  higher  the  motivation,  the  more  the  DA  was  employed 
and  the  more  of  its  capabilities  were  exploited.  In  general  this 
motivation  was  derived  from  desire  to  do  the  best  job  possible  and 
from  success  in  using  HF.  A  highly  motivated  user  wanted  to  exploit 
PROPHET  to  its  fullest.  He  knew  the  propagation  medium,  the 
equipment  he  was  using  and  how  to  use  PROPHET.  The  HF  signals 
problems  were  viewed  as  a  challenge  to  be  overcome.  Consequently 
this  persons  efforts  to  use  PROPHET  were  successful.  This 
individual  soon  acquired  considerable  expertise  in  using  PROPHET 
and  soon  became  a  "key"  person  in  this  aspect  of  normal  operations. 
But  eventually  this  "key"  individual  would  transfer  and,  often,  was 
replaced  by  person  who  could  be  considered  an  "antithesis"  to  the 
"key"  person  -  a  person  with  little  motivation  and  little  or  no 
experience  in  using  the  HF  medium.  If  this  user,  who  would  rather 
guess  than  do  any  real  problem  solving,  attempted  to  use  PROPHET 
for  frequency  planning  at  all,  the  attempt  was  half-hearted  with 
a  low  probability  of  success.  This  self-inflicted  bad  experience 
then  snowballed,  and  served  as  an  excuse  to  not  use  PROPHET,  or  any 
decision  aid,  again.  This,  in  turn,  caused  a  reduced  reliance  on 


mission  and  the  DA  system's  recommendations  as  to  sensor 
employment.  However  the  final  result  did  reflect  consideration  of 
such  complex  factors  as  propagation,  solar  activity,  network 
geometry,  and  historical  sensor  performance.  The  knowledge-based 
architecture  allowed  the  system  to  routinely  access  and  peruse 
large  databases  for  each  event.  Additional  features  were  included 
to  allow  the  user  to  intervene  at  any  level,  depending  on 
motivation  and  experience. 

CONCLUSIONS 

The  survey  of  FROFKET  users  indicated  that  the  successful 
deployment  of  stand-alone  a  decision  aids  is  vulnerable  to  the 
motivation  and  experience  of  the  user.  In  addition  its  employment 
is  heavily  influenced  by  how  readily  accessible  it  is  to  the 
intended  user  in  his  operational  environment. 

The  next  generation  decision  aid  will  be  a  hybrid  system  which  will 
combine  knowledge-based  expert  systems  with  traditional  algorithmic 
functions  such  as  those  in  PROPHET.  The  expert  systems  will 
specify,  setup,  and  control  complex  HP  connectivity  and  signal 
assessment  calculations  from  PROPHET-1 ike  models  which  and  carry 
on  the  query-response  dialogue  with  the  user.  To  ensure  that  it 
is  readily  accessible  operationally,  critical  issues  in  the  DA's 
early  design  are  (1)  what  mission  it  supports?  (2)  specifically 
who  the  users  are  and  how  much  time  they  can  devote  to  the  DA 
operations;  and  (3)  where  the  DA  will  be  physically  located. 

The  development  of  the  NOSC  Advanced  HFRLS  demonstration  terminal 
provided  lessons  that  are  critical  in  achieving  and  fully 
exploiting  the  next  generation  decision  aid.  It  was  shown  that: 

(1)  Algorithmic  signal-assessment  systems,  such  as  PROPHET,  can 
be  subordinated  and  tasked  by  a  knowledge  based  system.  This  makes 
the  employment  of  high-level  propagation  and  signal-warfare  models 
transparent  to  the  user. 

(2)  Subjective  information,  such  as  sensor  operator  experience, 
sensor  performance,  and  historical  trends  and  characteristics  can 
be  employed  in  the  decision  process  or  in  the  information 
displayed. 

(3)  Rule-based  expert-system  technology  allows  rapid,  intelligent 
perusal  of  immense  amounts  of  data  which  is  critical  in  the  control 
and  employment  of  a  large  communications  or  surveillance  network. 
It  further  provides  the  basis  for  "knowledge  cloning"  which  is 
critical  in  preserving  HF  technology. 
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Decision  aids,  such  as  PROPHET,  are  based  upon  deterministic 
calculations.  The  problems  amenable  to  PROPHET  for  analysis  are 
those  for  which  technical  methods  provide  explicit  solutions. 
However,  many  other  HF/VHF  radio  operations  problems  are  not 
amenable  to  such  explicit  analysis.  An  example  of  this  latter 
situation  is  the  prediction  of  when  and  where  sporadic-E  will 
occur.  Sporadic-E  has  a  profound  effect  on  both  HF  and  VHF 
systems  but  can  only  be  described  in  qualitative  terms  (e.g.it 
occurs  in  May  and  June  at  local  Sunset  and  drifts  from  the 
southeast  to  northwest) .  One  of  the  most  promising  methods  of 
being  able  to  deal  with  problems  that  don't  lend  themselves  to 
deterministic  solution  is  the  Expert  System.  An  Expert  System  is 
a  computerized  implementation  of  the  analytical  processes  that  a 
human  expert  employs  in  his  profession.  The  major  elements  of  an 
Expert  System  include  the  "knowledge  base"  and  the  "inference 
engine".  The  "knowledge  base  represents  the  body  of  information 
relevant  to  the  problem  under  scrutiny  and  may  contain 
"subjective"  rules  of  the  sort  used  by  human  experts  based  on 
experience;  from  these  rules,  the  system  can  construct  heuristic 
solutions.  The  "inference  engine"  contains  the  inference  and 
control  strategies  that  are  used  to  evaluate  the  knowledge  base  and 
synthesize  solutions.  Expert  systems  also  have  the  feature  of  being 
"knowledge  clones"  which  serves  to  preserve  expertise  in  various 
technologies.  This  is  particularly  needed  in  the  preservation  of 
HF  technology,  which  has  significantly  eroded  since  the  middle 
1960's. 

In  1987/88,  developmental  work  was  conducted  at  the  Naval  Ocean 
Systems  Center  to  combine  the  capabilities  of  presently-available 
knowledge-based  expert  system  technology  and  algorithmic  HF 
propagation-assessment  systems  such  as  PROPHET.  A  module,  called 
the  HF  Radiolocation  System  (HFRLS) ,  was  developed  to  test  and 
demonstrate  this  hybrid  concept.  The  initial  software  test 
vehicle  was  to  perform  resource  management  on  a  hypothetical 
network  of  dissimilar  HF  radiolocation  sensors.  These  included 
shore-based  wide-aperture  HFDF  systems,  shipboard  HFDF  systems, 
single  site  locator  systems  and  receive-only  HF  acquisition 
systems.  The  purpose  of  this  system  was  to  perform  (1)  scenario 
assessment,  (2)  resource  availability  and  capability  assessment, 
(3)  tasking  to  sensor  subnets,  and  (4)  assessment  of  the  decision 
on  whether  or  not  to  select  a  particular  sensor.  The  system  was 
structured  so  that  the  operator  would  start  to  converse  with  the 
system  at  the  start  of  an  event  and  would  monitor  progress  up 
through  the  final  solution  and  output  product. 

In-house  testing  of  HFRLS  indicated  that  using  an  "intelligent" 
man-machine  interface  greatly  enhances  the  employment  of  the  basic 
decision  aid.  The  major  propagation  models  in  PROPHET  were 
subordinated  to  the  HFRLS  system.  The  knowledge  based  system 
"knew"  when  propagation  data  was  necessary,  the  right  questions  to 
ask  PROPHET,  and  how  to  format  them  correctly.  This  was  an 
internal  dialogue  that  required  no  user  intervention,  so  the 
propagation  models  were  completely  transparent  to  the  user.  The 
user  saw  only  information  needed  to  perform  a  radiolocation 


the  decision  aid,  further  eroding  the  motivation.  When  allowed  to 
continue,  this  degenerative  cycle  eventually  eroded  all  the 
progress  gained  and  ultimately  the  DA  was  discarded  altogether. 

The  key  feature  in  defeating  this  negative  cycle  is  to  design  a 
user  interface  that  will  assure  the  new  decision  aid  user  with  the 
same  high  level  of  success,  irrespective  of  motivational  level  or 
experience , 

The  second  major  fallacy  in  designing  and  employing  signal 
assessment  decision  aids  is  to  assume  there  will  be  sufficient  time 
and  manpower  in  the  operational  environment  for  them  to  be  used. 
Generally  this  is  not  the  case,  especially  on  afloat  or  airborne 
platforms.  As  an  example,  consider  the  initial  deployment  of 
Classic  PROPHET  II  which  was  designed  to  support  OUTBOARD  HFDF 
operations.  The  module  was  designed  specifically  to  meet  the  needs 
of  OUTBOARD  operators  in  predicting  periods  when  signals  of 
interest  could  be  expected  to  be  heard.  Ships  personnel  were 
thoroughly  trained  in  using  it  on  a  stand-alone  personal  computer. 
However,  when  the  ship  got  underway,  the  intended  user's  time  was 
fully  consumed  manning  their  primary  positions.  There  was  no  extra 
time  to  perform  the  necessary  supporting  calculations  off  line, 
even  though  the  dedicated  computer  was  physically  only  few  feet 
away.  Although  the  capability  was  fully  functional,  its  stand¬ 
alone  location  made  it  impractical  for  the  user  to  access  it. 

This  experience  indicated  that  a  signal  assessment  decision  aid 
must  be  embedded  into  the  system  it  is  supporting. 


THE  NEXT  GENERATION  DECISION  AID 

New  requirements  have  been  proposed  for  the  next  generation  of 
signal  assessment  decision  aid.  The  proposed  design  was  influenced 
by  the  experiences  that  were  just  discussed.  It  must  be  able  to 
achieve  the  same  high  level  of  performance  or  productivity 
regardless  of  the  experience  and  /or  motivation  of  the  user,  and 
must  be  embedded  into  the  system  it  is  supporting. 

The  interface  of  the  next  generation  DA  must  allow  the  user  to 
converse  with  the  system  in  plain  language  or  in  the  jargon  of  the 
specific  mission.  Expanding  telecommunications  will  make  larger 
amounts  of  information  available  to  problem  solving  and  the  DA  must 
have  the  ability  to  peruse  these  data.  Complex  scientific 
calculations,  such  as  those  used  for  propagation  and  solar- 
terrestrial  relationships  will  be  performed  in  the  computer 
background,  outside  the  user  purview.  In  many  cases,  very 
complicated  models,  such  as  those  used  in  PROPHET,  will  be  totally 
transparent  to  the  user..., they  will  be  applied  to  a  users  problem 
in  a  normal  manner,  and  the  result  will  reflect  their  use,  but 
their  actual  operation  will  not  appear  in  the  user  system 
dialogue. 


